Moving picture coding method and a moving picture decoding method

ABSTRACT

A moving picture coding apparatus includes a motion estimation unit ( 101 ) for performing motion estimation by fixing the one of two reference pictures as a reference picture indicated by an inputted default reference picture number DefRefNo and a variable length coding unit ( 107 ) for performing variable length coding on coded residual data ERes, a prediction type PredType, a reference picture number RefNo 2  and motion vectors MV 1 , MV 2  on a block-by-block basis, and outputting them as coded moving picture data Str.

This application is a continuation of application Ser. No. 14/745,983,filed Jun. 22, 2015, which is a divisional of application Ser. No.13/495,318, filed Jun. 13, 2012, now abandoned, which is a divisional ofapplication Ser. No. 11/976,748, filed Oct. 26, 2007, now U.S. Pat. No.8,223,841, which is a continuation application of Ser. No. 10/480,932,filed Dec. 16, 2003, now U.S. Pat. No. 7,515,635, which is the NationalStage of International Application No. PCT/JP03/04806, filed Apr. 16,2003. The entire disclosures of the above-identified applications,including the specifications, drawings and claims are incorporatedherein by reference in their entirety.

TECHNICAL FIELD

The present invention relates to a method of coding and decoding movingpicture data as well as a recording medium on which a program forexecuting these methods as software is recorded.

BACKGROUND ART

In recent years, along with a development of multimedia applicationssuch as picture, audio and text, it has become general to handle allsorts of media in an integrated way. However, an information compressiontechnique for data is dispensable for its storage and transmission sincea digitalized picture contains an enormous amount of data. On the otherhand, a standardization of compression techniques is also important forinteroperating compressed picture data. The standards of picturecompression techniques include H.261, H.263 established by the ITU(International Telecommunication Union) and MPEG (Moving Picture ExpertsGroup)-1, MPEG-2 and MPEG-4 established by the ISO (InternationalOrganization for Standardization).

An inter-picture prediction which accompanies motion compensation can becited as a technique shared among these moving picture coding methods.In the motion compensation based on these moving picture coding methods,a picture of an input image is divided into blocks, each of which has apredetermined size, and a predictive image is generated for each blockusing motion vectors, respectively indicating a motion between pictures.The following predictions are employed for the inter-picture predictionaccording to the MPEG: a forward prediction for a prediction using asingle picture whose display time is earlier than that of a currentpicture to be coded; a backward prediction for a prediction using asingle picture whose display time is later than that of the currentpicture; a bi-directional prediction for a prediction using twopictures, that is, one picture whose display time is earlier than thecurrent picture and the other picture whose display time is later thanthat of the current picture (see reference, for example, ISO/IEC14496-2:1999(E) Information technology—coding of audio-visual objectsPart 2: Visual (1999-12-01) pp 150 7.6.7 Temporal prediction structure).

In the MPEG, a reference picture to be used is determined uniquelydepending on the type of inter-picture prediction and an arbitraryreference picture cannot be selected. In the meantime, a bi-directionalprediction which is expanded so that two arbitrary reference picturescan be selected out of a plurality of coded pictures stored in a picturememory regardless of the display time of the current picture is takenunder the consideration in the H.264 which is presently under theprocess of standardization by the ITU.

FIG. 1 is a block diagram showing a structure of a moving picture codingapparatus according to the H.264. The conventional moving picture codingapparatus shown in FIG. 1 is an apparatus for executing a moving picturecoding method which allows a selection of two arbitrary referencepictures from plural coded pictures when the inter-picture prediction isoperated.

This moving picture coding apparatus includes, as shown in FIG. 1, amotion estimation unit 301, a pixel interpolation unit 102, a subtractor103, a picture coding unit 104, a picture decoding unit 105, an adder106, a variable length coding unit 302, a multi-picture buffer 108 and aswitch 109.

The moving picture coding apparatus divides an inputted image data Imginto blocks and performs processing for each of the blocks. Thesubtractor 103 subtracts a predictive image data Pred from the imagedata Img inputted to the moving picture coding apparatus and outputs itas residual data Res. The picture coding unit 104 performs picturecoding processing such as orthogonal transformation and quantization onthe inputted residual data Res and outputs it as coded residual dataERes including quantized orthogonal transformed coefficients. Thepicture decoding unit 105 performs picture decoding processing such asinverse quantization and inverse orthogonal transformation on theinputted coded residual data ERes and outputs it as decoded residualdata DRes. The adder 106 adds the decoded residual data DRes to thepredictive image data Pred and outputs it as reconstructed image dataRecon. Out of the reconstructed image data Recon, the data having thepossibility to be used for reference in the subsequent inter-pictureprediction is stored in the multi-picture buffer 108.

Here, an interpolation prediction using two reference pictures performedby the conventional moving picture coding apparatus is described withreference to FIG. 2. FIG. 2 is a conceptual diagram of the interpolationprediction using plural reference pictures. Here, a picture Pic is acurrent picture to be coded. Pictures FwRef1˜FwRef3 represent codedpictures respectively having a display time earlier than that of thecurrent picture whereas pictures BwRef1˜BwRef3 represent coded picturesrespectively having a display time later than that of the currentpicture. A block Blk1 is predicted using pixel values in a referenceblock RefBlk11 included in the picture FwRef3 whose display time isearlier than that of the current picture Pic and pixel values in areference block RefBlk12 included in the picture BwRef1 whose displaytime is later than that of the current picture Pic. A block Blk2 ispredicted using pixel values in reference blocks RefBlk21 and RefBlk22included in two pictures FwRef1 and FwRef2 respectively having a displaytime earlier than that of the current picture. A block Blk3 is predictedusing pixel values in reference blocks RefBlk31 and RefBlk32 included intwo pictures BwRef1 and BwRef2 respectively having a display time laterthan that of the current picture. Namely, a result of interpolatingpixels in the areas corresponding to the two reference blocks using apredetermined method such as the one using an average value isconsidered to be a predictive image. The characteristics of theconventional moving picture coding apparatus is to perform prediction ona block-by-block basis using arbitrary two reference pictures as shownin FIG. 2. A method for predicting with the use of two arbitraryreference pictures as described above is called “plural referencepicture interpolation prediction” hereinafter. The prediction methodincludes a method in which a block included in a single arbitrarypicture is used directly as a predictive image and the intra-pictureprediction other than the method of generating a predictive image usingthe pixel interpolation as described above, and it is possible to switchthe prediction method on a block-by-block basis.

The motion estimation unit 301 determines a prediction type for theblock, reference pictures and motion vectors to be used forinter-picture prediction performed on the inputted current block to becoded and outputs a prediction type PredType, reference picture numbersRefNo1, RefNo2, and motion vectors MV1, MV2. The motion estimation 301outputs two picture numbers and two motion vectors since two referencepictures are selected when plural reference picture interpolationprediction is operated. Here, the multi-picture buffer 108 outputs areference block RefBlk1 corresponding to the reference picture numberRefNo1 and the motion vector MV1 and a reference block RefBlk2corresponding to the reference picture number RefNo2 and the motionvector MV2. The pixel interpolation unit 102 performs interpolation forthe pixels with respect to the two reference blocks RefBlk1 and RefBlk2using average value and outputs it as an interpolated block RefPol. Onthe other hand, in the case of using an inter-picture prediction otherthan a plural reference picture interpolation prediction, the motionestimation unit 301 selects a single reference picture, and therefore,outputs a single reference picture number RefNo1 and a single motionvector MV1. In this case, the multi-picture buffer 108 outputs areference block RefBlk with respect to the reference picture numberRefNo1 and the motion vector MV1.

When the prediction type determined by the motion estimation unit 301indicates a plural reference picture interpolation prediction, theswitch 109 is switched to a “1” side and the interpolated block RefPolis used as a predictive image data Pred. When the prediction typePredType indicates an inter-picture prediction other than a pluralreference picture interpolation prediction, the switch SW11 is switchedto a “0” side and the reference block RefBlk is used as a predictiveimage data Pred. The variable length coding unit 302 performs variablelength coding on the coded residual data ERes, the prediction typePredType, the reference picture numbers RefNo1, RefNo2 and the motionvectors MV1, MV2 and then outputs them as coded moving picture dataStr0.

FIG. 3 is a conceptual diagram of a data format of coded moving pictureused by the conventional moving picture coding apparatus. Coded dataequivalent to a single picture, Picture, is composed of coded dataequivalent to a single block, Block, where each block composes apicture, and the like. Here, the coded data equivalent to a singleblock, Block, presents coded data of a block on which a plural referencepicture interpolation prediction is performed, and includes thereference picture numbers RefNo1, RefNo2, the motion vectors MV1, MV2,with respect to the two reference pictures, the prediction modePredType, and the like, in the coded data.

FIG. 4 is a block diagram showing a structure of the conventional movingpicture decoding apparatus. The moving picture decoding apparatusincludes, as shown in FIG. 4, a variable length decoding unit 601, amotion compensation unit 602, a picture decoding unit 404, an adder 405,a pixel interpolation unit 406, a multi-picture buffer 407 and a switch408.

The variable length decoding unit 601 performs variable length decodingon the inputted coded image data Str0 and outputs the coded residualdata ERes, the motion vectors MV1, MV2, the reference picture numbersRefNo1, RefNo2 and the prediction type PreType. The picture decodingunit 404 performs picture decoding processing such as inversequantization and inverse orthogonal transformation on the inputted codedresidual data ERes and outputs decoded residual data DRes. The adder 405adds the decoded residual data DRes to the predictive image data Predand outputs it as decoded image data DImg outside the moving picturedecoding apparatus. The multi-picture buffer 407 stores the decodedimage data DImg for inter-picture prediction.

The motion compensation unit 602 outputs reference picture numbersNRefNo1, NRefNo2 of the reference blocks necessary for inter-pictureprediction according to the prediction type PredType as well as themotion vectors MV1, MV2 and instructs the multi-picture buffer 407 tooutput the reference blocks. When the prediction type PredType indicatesa plural reference picture interpolation prediction, the multi-picturebuffer 407 outputs the reference block RefBlk1 corresponding to thereference picture number NRefNo1 and the motion vector NMV1 as well asthe reference block RefBlk2 corresponding to the reference picturenumber NRefNo2 and the motion vector NMV2. The pixel interpolation unit406 interpolates the pixels in the two reference blocks RefBlk1 andRefBlk2 using the average value. On the other hand, when the predictiontype PredType indicates an inter-picture prediction method other than aplural reference picture interpolation prediction, the multi-picturebuffer 407 outputs the reference block RefBlk corresponding to thereference picture number NRefNo1 and the motion vector NMV1.

When the prediction type PreType indicates a plural reference pictureinterpolation prediction, the switch 408 is switched to a “0” side andan interpolated block RefPol is used as a predictive image data Pred.Thus, the moving picture decoding apparatus decodes the coded movingpicture data Str0 through the processing described above and outputs itas decoded image data DImg.

Meanwhile, under the moving picture coding method based on the MPEG-4, aplural reference picture interpolation prediction method called “directmode” is defined for a picture type, called “bi-directional predictivepicture”, employing a plural reference picture interpolation prediction.It is defined as a method to abbreviate the motion vectors and thereference picture numbers included in the coded data of the block bycalculating the motion vectors with respect to two reference picturesused for the generation of the predictive image by means ofinterpolation using the coded motion vectors.

FIG. 5 is an illustration for a case of using the direct mode defined inthe MPEG-4. Here, a picture Pic represents a current picture to becoded, a picture Ref1 represents a reference picture whose display timeis earlier than that of the current picture Pic and a picture Ref2represents a reference picture whose display time is later than that ofthe current picture Pic whereas a block Blk represents a current blockto be coded and a block Blk0 represents a block whose position is sameas that of the current block Blk in the reference picture Ref2. A motionvector MV01 represents a forward reference motion vector using thepicture Ref1 as a reference picture for coding the block Blk0, a motionvector MV1 represents a motion vector of the current block with respectto the reference picture Ref1, a motion vector MV2 represents a motionvector of the current block with respect to the reference picture Ref2,a block RefBlk1 represents a reference block to be referred to by themotion vector MV1 and a block RefBlk2 represents a reference block to bereferred to by the motion vector MV2.

As for the two pictures to be used for reference by the current blockBlk, the picture Ref2 whose display time is later than and is closest tothe current picture is used as a backward reference picture whereas thepicture Ref1, which has been used for reference by the block Blk0 at thetime of coding, is used as a forward reference picture.

For the calculation of the motion vectors, it is assumed that either themotion is constant or no motions are found in comparing the pictures.Here, with an assumption that a differential value between the displaytime of the current picture Pic and that of the reference picture Ref1is TRD1, a differential value between the display time of the referencepicture Ref1 and that of the reference picture Ref2 is TRD2, and adifferential value between the display time of the current picture Picand that of the reference picture Ref2 is TRD3, the motion vectors MV1and MV2 to be used for coding the current block can be calculatedrespectively using the following equations:

MV1=MV01×(TRD1/TRD2)  (Equation A)

MV2=−MV01×(TRD3/TRD2)  (Equation B)

Using the above method, the reference pictures and the motion vectors inthe case of using a direct mode can be determined. The processing in thecase of using a direct mode as described above, performed by the movingpicture coding apparatus, is executed by the motion estimation unit 301shown in the block diagram illustrating the conventional moving picturecoding apparatus in FIG. 1. The processing for the case of using adirect mode described above, performed by the moving picture decodingapparatus, is executed by the motion compensation unit 602 shown in theblock diagram illustrating the conventional moving picture decodingapparatus in FIG. 4.

When a moving picture, in which a motion between the pictures is small,is inter-picture coded, a predictive error between the pictures becomevery small and most of the coded residual data ERes become “0” byperforming picture coding processing such as quantization. A case inwhich the entire coded residual data ERes resulted from theinter-picture prediction using the reference pictures and the motionvectors of the current block is “0” in the coding in which the motionvectors and the reference pictures are determined using a predeterminedmethod without coding them, as in the case of using a direct mode asdescribed above, is defined as one of the prediction types PredTypecalled “skip mode”. In using a skip mode, only the prediction typePredType indicating the skip mode is transmitted, therefore, coding of ablock requires a very small code amount. The efficiency of coding can befurther improved by assigning variable length code that is shorter thanother prediction types to this skip mode or by run-length coding thenumber of consecutive blocks used for the skip mode.

In the H.264 described above, “skip mode” is defined as a case in whichthe entire coded residual data equivalent to a single block obtained bythe inter-picture prediction using a direct mode is assumed to be “0”.The following processing is performed when a block is coded using a skipmode by the moving picture coding apparatus shown in FIG. 1. The motionestimation unit 301 outputs the reference picture numbers RefNo1,RefNo2, the motion vectors MV1, MV2 as well as the prediction typePredType indicating a skip mode. The variable length coding unit 302performs variable length coding only for the prediction type PredTypeand outputs it as coded moving picture data Str0 through the processingexplained above, when the prediction type PredType indicates a skipmode. The following processing is performed when the coded data of theblock coded using a skip mode is inputted to the moving picture decodingapparatus shown in FIG. 4. The variable length decoding unit 601performs variable length decoding on the prediction type PredType. Whenthe prediction type PredType indicates a skip mode, the motioncompensation unit 602 outputs, through the processing operated in thecase of direct mode explained above, the reference picture numbersNRefNo1, NRefNo2, the motion vectors NMV1, NMV2 as well as theprediction type PredType indicating a skip mode.

In the H.264 as described above, arbitrary reference pictures can beselected out of a plurality of coded pictures regardless of the displaytime of the current picture. However, the arbitrary reference picturesare selected by performing motion estimation for the plurality of thecoded pictures in this case, therefore, the processing burden caused bythe motion estimation becomes very large. The plural reference pictureinterpolation prediction also contains a problem of degrading the codingefficiency since it requires coding of reference picture numbers andmotion vectors for every two reference pictures.

Furthermore, when inter-picture prediction is performed for a pictureusing a picture whose display time is later than that of the currentpicture as a reference picture, as in the case of bi-directionalprediction described in the conventional technique, the picture has tobe coded in an order different from a display order, which causes adelay. In a case of real time communication such as a videophone,bi-directional predictive pictures cannot be used because of the delay.In the H.264, however, two arbitrary reference pictures can be selectedregardless of display order information, therefore, the delay caused bycoding can be eliminated by performing a plural reference pictureinterpolation prediction with a selection of two pictures respectivelyhaving a display time which is earlier than that of the current picture.However, the picture whose display time is later than that of thecurrent picture is not stored in the multi-picture buffer, therefore,the direct mode conventionally used for determining the motion vectorsusing the picture whose display time is later than that of the currentpicture as described above cannot be employed.

DISCLOSURE OF INVENTION

The present invention is conceived in view of above circumstances, andaims to provide a moving picture coding method and a moving picturedecoding method for realizing an effective coding as well as a reductionof the processing burden when a plural reference picture interpolationprediction is performed.

In order to achieve the above objects, the moving picture coding methodaccording to the present invention codes each picture composing an inputmoving picture on a block-by-block basis and comprises: a commonreference picture determination step of determining a picture shared forreference among a plurality of blocks on which coding is performed withreference to a coded picture; a predictive image generation step ofgenerating a predictive image using the common reference picture; and acoding step of coding a current block to be coded using the predictiveimage.

Thus, when the predictive image is generated using a reference picture,the processing burden can be reduced since the processing for selectingon a block-by-block basis a picture to be used as a reference picturefrom among a plurality of coded pictures is not required. The coding ofthis reference picture on a block-by-block basis is not necessary.Therefore, the bit amount can be reduced. In general, it is highlypossible that most of the blocks in the image data select the samepicture as an optimal reference picture. Therefore, it is possible toreduce the processing burden while maintaining a high coding efficiencyby sharing a reference picture on a picture-by-picture basis, forinstance.

Also, the moving picture coding method according to the presentinvention codes each picture composing an input moving picture on ablock-by-block basis and comprises: a common reference picturedetermination step of determining a first picture to be shared forreference picture among a plurality of blocks on which coding isperformed with reference to two coded pictures; a predictive imagegeneration step of generating a predictive image with reference to thefirst picture and a second picture selected on a block-by-block basisfrom among coded pictures; and a coding step of coding a current blockto be coded using the predictive image.

Thus, when the predictive picture is generated with reference to tworeference pictures, the processing burden can be reduced since theprocessing for selecting a single picture on a block-by-block basis asthe one reference picture from among a plurality of coded pictures isnot necessary. The bit amount can be also reduced since the coding ofthis reference picture on a block-by-block basis is not necessary.Generally speaking, it is highly possible that most of the blocks in theimage data select the same picture as an optimal reference picture.Therefore, it is possible to reduce the processing burden whilemaintaining a high coding efficiency by sharing one of the referencepictures on a picture-by-picture basis, for instance.

Here, the moving picture coding method may further comprise aninformation description step of describing information for specifyingthe common reference picture in a common information area assigned forthe plurality of blocks in coded moving picture data to be generated.

Thus, the information for specifying the common reference picture can bedescribed in the coded moving picture data and then outputted,therefore, the reference picture can be certainly specified when thecoded moving picture data is decoded.

The moving picture decoding method according to the present inventiondecodes coded moving picture data obtained by coding each picture on ablock-by-block basis and comprises: a common reference picturedetermination step of determining a picture shared for reference among aplurality of blocks on which decoding is performed with reference to adecoded picture; a predictive image generation step of generating apredictive image using the common reference picture; and a decoding stepof decoding a current block to be decoded using the predictive image.

Thus, the coded moving picture data, which is coded with reference to acommon reference picture and then outputted, can be decoded properly inthe decoding processing.

Also, the moving picture decoding method according to the presentinvention decodes coded moving picture data obtained by coding eachpicture on a block-by-block basis and comprises: a common referencepicture determination step of generating a predictive image withreference to the first picture and a second picture selected on ablock-by-block basis from among decoded pictures; and a decoding step ofdecoding a current block to be decoded using the predictive image.

Thus, the moving picture data, which is coded with reference to a commonreference picture and a reference picture used on a block-by-blockbasis, can be decoded properly in the decoding processing.

Here, the moving picture decoding method may further comprise aninformation extraction step of extracting information for specifying thecommon reference picture in a common information area assigned for theplurality of blocks in the coded moving picture data.

Thus, the information for specifying the common reference picture can beextracted from the coded moving picture, and thereby, the referencepicture can surely be specified.

The present invention can be realized not only as the moving picturecoding method and the moving picture decoding method as described above,but also as a moving picture coding apparatus and a moving picturedecoding apparatus having characteristic steps included in the movingpicture coding method and the moving picture decoding method as units.It can be also realized as a program having a computer execute thesesteps or as coded moving picture data that is coded with the use of themoving picture coding method. Needless to say, such program and codedmoving picture data can be distributed via a recording medium such as aCD-ROM and a transmission medium such as an Internet.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a structure of a conventional movingpicture coding apparatus.

FIG. 2 is a conceptual diagram showing an interpolation prediction usinga plurality of reference pictures.

FIG. 3 is a conceptual diagram showing a data format of a coded movingpicture employed by the conventional moving picture coding apparatus.

FIG. 4 is a block diagram showing a structure of a conventional movingpicture decoding apparatus.

FIG. 5 is an illustration for a conventional direct mode.

FIG. 6 is a block diagram showing a structure of a moving picture codingapparatus according to a first embodiment.

FIG. 7 is a conceptual diagram showing a data format of a coded movingpicture according to the first embodiment.

FIG. 8 is a block diagram showing a structure of a moving picturedecoding apparatus according to a second embodiment.

FIG. 9 is a block diagram showing a structure of a moving picture codingapparatus according to a third embodiment.

FIG. 10 is a conceptual diagram showing a data format of a coded movingpicture according to the third embodiment.

FIG. 11 is a block diagram showing a structure of a variation of themoving picture coding apparatus according to the third embodiment.

FIG. 12 is a conceptual diagram showing a data format of a coded movingpicture according to the variation of the third embodiment.

FIG. 13 is a block diagram showing a variation of the moving picturecoding apparatus according to the third embodiment.

FIG. 14 is a block diagram showing a structure of a moving picturedecoding apparatus according to a fourth embodiment.

FIG. 15 is a block diagram showing a structure of a variation of themoving picture decoding apparatus according to the fourth embodiment.

FIG. 16 is an illustration of a direct mode according to a fifthembodiment, with the use of plural reference pictures respectivelyhaving information on a display time which is earlier than that of acurrent picture.

FIG. 17 is an illustration of a direct mode according to the fifthembodiment, with the use of plural reference pictures respectivelyhaving information on a display time which is later than that of thecurrent picture.

FIG. 18 is an illustration of an inter-picture prediction using a skipmode according to a sixth embodiment.

FIGS. 19A, 19B and 19C are illustrations of a recording medium forstoring a program for realizing the moving picture coding method or themoving picture decoding method according to each of the embodiments in acomputer system. FIG. 19A is an illustration showing an example of aphysical format of a flexible disk which is a main body of a storingmedium. FIG. 19B is an illustration showing a full appearance of theflexible disk, a structure at cross section and the flexible diskitself. FIG. 19C is an illustration showing a configuration forrecording and reproducing the program on the flexible disk FD.

FIG. 20 is a block diagram showing a whole configuration of a contentdelivery system for realizing a content delivery service.

FIG. 21 is a sketch showing an example of a cell phone.

FIG. 22 is a block diagram showing an internal structure of the cellphone.

FIG. 23 is a block diagram showing a whole configuration of a digitalbroadcasting system.

BEST MODE FOR CARRYING OUT THE INVENTION First Embodiment

FIG. 6 is a block diagram showing a structure of a moving picture codingapparatus according to the first embodiment. The same marks are put forthe units and the data operating in the same manner as described in theblock diagram showing a structure of a conventional moving picturecoding apparatus in FIG. 1 and the description will be abbreviated. Itis possible for the moving picture coding apparatus and the movingpicture decoding apparatus according to each embodiment described belowto switch, on a block-by-block basis, between the following predictionmethods: a method of generating a predictive image by pixelinterpolation using two reference pictures (a plural reference pictureinterpolation prediction); a method of using a block included in asingle arbitrary picture directly as a predictive image; a method ofgenerating a predictive image using an intra-picture prediction.

The moving picture coding apparatus is an apparatus for dividing aninputted picture data Img into blocks and performing coding on each ofthe blocks, and includes a motion estimation unit 101, the pixelinterpolation unit 102, the subtractor 103, the picture coding unit 104,the picture decoding unit 105, the adder 106, a variable length codingunit 107, the multi-picture buffer 108 and the switch 109.

A default reference picture number DefRefNo indicating the one of thereference pictures to be used for a block that is coded using a pluralreference picture interpolation prediction is inputted to the movingpicture coding apparatus. The motion estimation unit 101 performs motionestimation by fixing one of two reference pictures as the referencepicture indicated by the inputted default reference picture numberDefRefNo, when a plural reference picture interpolation prediction isperformed. The reference picture number RefNo1 outputted by the motionestimation unit 101 therefore indicates the same value as indicated bythe default reference picture number DefRefNo. The variable lengthcoding unit 107 performs variable length coding for the coded residualdata ERes, the prediction type PredType, the reference picture numberRefNo2, the motion vectors MV1, MV2, the default reference picturenumber DefRefNo and outputs them as coded moving picture data Str.

The following describes an operation performed by the moving picturecoding apparatus constructed as above, when the prediction type of thecurrent block is a plural reference picture interpolation prediction.

The inputted image data Img is inputted to the motion estimation unit101 and the subtractor 103 on a block-by-block basis.

The motion estimation unit 101 determines a prediction type of theinputted current block and outputs the prediction type to the switch 109and the variable length coding unit 107. When the determined predictiontype PredType is a plural reference picture interpolation prediction,the motion estimation unit 101 determines the one of the two referencepictures as the reference picture indicated by the inputted defaultreference picture number DefRefNo and determines respectively the otherreference picture and motion vectors MV1 and MV2 with respect to thesetwo reference pictures. The motion estimation unit 101 then outputs areference picture number RefNo2 and the motion vectors MV1 and MV2 tothe multi-picture buffer 108 and the variable length coding unit 107 aswell as the reference picture number RefNo1 to the multi-picture buffer108. The default reference picture number DefRefNo may be outputted fromthe motion estimation 101 to the variable length coding unit 107.

Next, the multi-picture buffer 108 outputs a reference block RefBlk1corresponding to the reference picture number RefNo1 and the motionvector MV1 as well as a reference block RefBlk2 corresponding to thereference picture number RefNo2 and the motion vector MV2 to the pixelinterpolation unit 102. The pixel interpolation 102 interpolates pixelswith respect to the two reference blocks RefBlk1 and RefBlk2 with theuse of an average value and outputs it as an interpolated block RefPol.Here, the prediction type determined by the motion estimation unit 101is a plural reference picture interpolation prediction, the switch 109is switched to a “1” side and the interpolated block RefPol is outputtedas predictive image data Pred to the subtractor 103 and the adder 106.

The subtractor 103 subtracts the predictive image data Pred from theinputted image data Img and outputs it as residual data Res to thepicture coding unit 104. The picture coding unit 104 performs picturecoding processing such as orthogonal transformation and quantization onthe inputted residual data Res and outputs it as coded residual dataERes to the picture decoding unit 105 and the variable length codingunit 107. The picture decoding unit 105 performs picture decodingprocessing such as inverse quantization and inverse orthogonaltransformation on the inputted coded residual data ERes and outputs itas decoded residual data DRes to the adder 106. The adder 106 adds thedecoded residual data DRes to the predictive image data Pred and outputsit as reconstructed image data Recon. The data having the possibility tobe used for reference in the subsequent inter-picture prediction out ofthe reconstructed data Recon is stored in the multi-picture buffer 108.

The variable length coding unit 107 performs variable length coding forthe inputted coded residual data ERes, the prediction type PredType, thereference picture number RefNo2 and the motion vectors MV1, MV2 for eachblock and outputs them as coded moving picture data Str.

For the picture indicated by the default reference picture numberDefRefNo, an arbitrary picture can be selected from the pictures storedin the multi-picture buffer 108. For example, a coded picture havingdisplay order information which is the closest to that of the currentpicture, a coded picture having display order information which is priorto and the closest to that of the current picture, a coded picturehaving display order information which is subsequent to and the closestto that of the current picture, and the like, in the multi-picturebuffer 108, are conceivable.

Similarly, a picture that is the closest to the current picture in acoding order, a picture having display order information which is priorto that of the current picture and a coding order which is the closestto that of the current picture, a picture having display orderinformation which is prior to that of the current picture and a codingorder which is the closest to that of the current picture, a picturehaving display order information which is subsequent to that of thecurrent picture and a coding order which is the closest to that of thecurrent picture, and the like, are also conceivable.

FIG. 7 is a conceptual diagram showing a data format of a coded movingpicture according to the first embodiment. The same marks are put forthe same data as described in the conceptual diagram showing the dataformat of the coded moving picture employed by the conventional movingpicture coding apparatus shown in FIG. 3, and the description will beabbreviated. The difference between the data format of the coded movingpicture according to the present embodiment and the one employed by theconventional moving picture coding apparatus is that a default referencepicture number DefRefNo is included for each picture and that only asingle data for the reference picture number is included in the codeddata of the block that is coded using a plural reference pictureinterpolation prediction.

According to the present embodiment as described above, the codingefficiency can be improved since the fixed reference picture number doesnot need to be coded on a block-by-block basis. This is because anarbitrary picture is selected for the one reference picture on ablock-by-block basis from among plural coded pictures, and the otherreference picture can be fixed as a picture among plural coded pictureson a picture-by-picture basis.

In the present embodiment, as a method of specifying a default referencepicture, the picture numbers are assigned to the pictures, however, thepresent invention shall not be limited to this. For example, it ispossible to specify a default reference picture either using a relativedifferential value between the picture number possessed by the currentpicture and the picture number possessed by the picture selected as adefault reference picture or using information such as a commandindicating a default reference picture.

In the present embodiment, only one reference picture is specified as adefault reference picture, however, both of the two reference picturenumbers in the coded data of the block can be abbreviated by coding twodefault reference picture numbers.

Also, in the present embodiment, the description is provided for theplural reference picture interpolation prediction for generating apredictive image by pixel interpolation using two reference pictures.However, a case of single reference picture interpolation predictionusing a block included in an arbitrary single picture as a predictiveimage can be handled in the same manner. In this case, there is no needto describe the reference picture information for each block, andtherefore, the reference picture information is described only in acommon information area.

The default reference picture numbers are coded on a picture-by-picturebasis in the present embodiment, however, they may be coded using asyntax structure which stores a single default reference picture numberfor every plural pictures or they may be coded using a syntax structurewhich stores a single default reference picture number for a syntaxstructure lower than a picture such as a macroblock which is composed ofplural blocks or a slice which is made up of plural macroblocks, or thelike.

Second Embodiment

FIG. 8 is a block diagram showing a moving picture decoding apparatusaccording to the second embodiment of the present invention. The samemarks are put for the units and the data operating in the same manner asillustrated in the block diagram showing a structure of the conventionalmoving picture decoding apparatus in FIG. 4, and the description will beabbreviated. The difference between the moving picture decodingapparatus of the present embodiment and the conventional one shown inFIG. 4 is that a default reference picture number buffer 402 is added tothe former.

The moving picture decoding apparatus, as shown in FIG. 8, includes avariable length decoding unit 401, a default reference picture numberbuffer 402, a motion compensation unit 403, a picture decoding unit 404,an adder 405, a pixel interpolation unit 406, a multi-picture buffer 407and a switch 408.

The variable length decoding unit 401 performs variable length decodingon the inputted coded moving picture data Str and outputs coded residualdata ERes, a prediction type PredType, a reference picture numberRefNo2, motion vectors MV1 and MV2, a default reference picture numberDefRefNo. The decoded default reference picture number DefRefNo needs tobe shared among plural blocks so that it is stored in the defaultreference picture number buffer 402. The default reference picturenumber DefRefNo stored in the default reference picture number buffer402 is inputted as a reference picture number RefNo1 in the motioncompensation unit 403.

The following describes an operation of the moving picture decodingapparatus constructed as above when a prediction type of a current blockto be decoded is a plural reference picture interpolation prediction.

The coded moving picture data Str is inputted to the variable lengthdecoding unit 401. The variable length decoding unit 401 performsvariable length decoding on the inputted coded moving picture data Strand outputs respectively as follows: the coded residual data ERes to thepicture decoding unit 404; the reference picture number RefNo2 and themotion vectors MV1, MV2 to the motion compensation unit 403; theprediction type PredType to the motion compensation unit 403 and theswitch 408; and the default reference picture number DefRefNo to thedefault reference picture number buffer 402. The default referencepicture number buffer 402 outputs the stored default reference picturenumber DefRefNo as a reference picture number RefNo1 to the motioncompensation unit 403.

Since the prediction type PredType is a plural reference pictureinterpolation prediction, the motion compensation unit 403 outputs, tothe multi-picture buffer 407, the reference picture number NRefNo1inputted by the default reference picture number buffer 402 and thereference picture number RefNo2 and the motion vectors MV1, MV2 inputtedby the variable length decoding unit 401 and instructs an output of thereference blocks. The multi-picture buffer 407 outputs, to the pixelinterpolation unit 406, the reference block RefBlk1 corresponding to thereference picture number NRefNo1 and the motion vector NMV1 and thereference block RefBlk2 corresponding to the reference picture numberNRefNo2 and the motion vector NMV2. The pixel interpolation unit 406interpolates the pixel values with respect to the two reference blocksRefBlk1 and RefBlk2 using average value and outputs it as aninterpolated block RefPol. Here, since the prediction type is a pluralreference picture interpolation prediction, the switch 408 is switchedto a “0” side and the interpolated block RefPol is thereby outputted asa predictive image data Pred to the adder 405.

On the other hand, the picture decoding unit 404 to which the codedresidual data ERes is inputted performs picture decoding processing suchas inverse quantization and inverse orthogonal transformation andoutputs decoded residual data DRes to the adder 405. The adder 405 addsthe decoded residual data DRes to the predictive image data Pred andoutputs it as decoded image data DImg outside the moving picturedecoding apparatus. The multi-picture buffer 407 stores the decodedimage data DImg for inter-picture prediction. The moving picturedecoding apparatus decodes the coded moving picture data Str throughsuch processing and outputs it as the decoded image data DImg.

According to the present embodiment as described above, it is possibleto decode properly the coded moving picture data Str which is coded bythe moving picture coding apparatus using the moving picture codingmethod of the present invention described in the first embodiment.

Third Embodiment

FIG. 9 is a block diagram showing a moving picture coding apparatusaccording to the third embodiment of the present invention. The samemarks are put for the units and the data operating in the same manner asshown in the block diagram illustrating the moving picture codingapparatus according to the first embodiment in FIG. 6, and thedescription will be abbreviated.

The moving picture coding apparatus of the present embodiment includes adefault reference picture number generation unit 201 in addition to thestructure shown in the first embodiment. The default reference picturenumber generation unit 201 generates a default reference picture numberDefRefNo using a predetermined method and outputs it to the motionestimation unit 101. The motion estimation unit 101 performs motionestimation, by fixing the one of two reference pictures as the referencepicture indicated by the inputted default reference picture numberDefRefNo, when the plural reference picture interpolation prediction isperformed as in the case of the moving picture coding apparatusaccording to the first embodiment. The variable length coding unit 202performs variable length coding on the coded residual data ERes, theprediction type PredType, the reference picture number RefNo2 and themotion vectors MV1, MV2 and outputs them as coded moving picture dataStr2.

For example, the following methods are available as a method ofgenerating a default reference picture number DefRefNo employed by thedefault reference picture number generation unit 201. The first methodis to determine, as a default reference picture number DefRefNo, apicture number indicating a picture having display order informationwhich is the closest to that of the current picture out of the codedpictures stored in the multi-picture buffer 108. The second method is todetermine, as a default reference picture number DefRefNo, a picturenumber indicating a picture having display order information which isprior to and the closest to that of the current picture out of the codedpictures stored in the multi-frame buffer 108. The third method is todetermine, as a default reference picture number DefRefNo, a picturenumber indicating a picture having display order information which issubsequent to and is the closest to that of the current picture out ofthe coded pictures stored in the multi-picture buffer 108. The fourthmethod is to determine, as a default reference picture number DefRefNo,a picture number indicating a picture whose coding order is the closestto that of the current picture out of the coded pictures stored in themulti-picture buffer 108. The fifth method is to determine, as a defaultreference picture number DefRefNo, a picture number indicating a picturewhich has display order information prior to that of the current pictureand whose coding order is the closest to that of the current picture outof the coded pictures stored in the multi-picture buffer 108. The sixthmethod is to determine, as a default reference picture number DefRefNo,a picture number indicating a picture which has display orderinformation subsequent to that of the current picture and whose codingorder is the closest to that of the current picture out of the codedpictures stored in the multi-picture buffer 108.

The data format of the coded moving picture used by the moving picturecoding apparatus according to the present embodiment is as shown in FIG.10, from which the default reference picture number DefRefNo shown inthe data format of the coded moving picture shown in FIG. 7 is omitted.Therefore, a default reference picture number DefRefNo does not need tobe coded, which improves the coding efficiency.

In the above-mentioned embodiment, a method of realizing the codingwithout describing information on the default reference picture at allon a data format by fixing a method to any arbitrary one for determininga default reference picture is explained. It is, however, possible toswitch between the methods to determine a default reference picture on apicture-by-picture basis. For example, this can be realized by codingeither of the following identifiers: an identifier indicating a methodof selecting, as a default reference picture, a picture having displaytime information which is the closest to that of the current picture outof the coded pictures stored in the multi-picture buffer; an identifierindicating a method of selecting, as a default reference picture, apicture having display time information which is prior to and is theclosest to that of the current picture out of the coded pictures storedin the multi-picture buffer; and an identifier indicating a method ofselecting, as a default reference picture, a picture having informationon a display time which is subsequent to and is the closest to that ofthe current picture out of the coded pictures stored in themulti-picture buffer.

FIG. 11 is a block diagram showing the moving picture coding apparatusused for this case. The default reference picture number generation unit203 outputs an identifier Ident indicating a method of selecting adefault reference picture to the variable length coding unit 204, asshown in FIG. 11. The variable length coding unit 204 performs variablelength coding on the coded residual data ERes, the prediction typePredType, the reference picture RefNo2, the motion vectors MV1 and MV2as well as the identifier Ident and outputs them as coded moving picturedata Str3. The data format for this case includes an identifier Identfor indicating a method of selecting a default reference picture asshown in FIG. 12 instead of the default reference picture numberDefRefNo that is information directly specifying a default referencepicture as shown in FIG. 7.

Similarly, it is possible to code an identifier indicating a method ofselecting, as a default reference picture, a picture whose coding orderis the closest to that of the current picture out of the coded picturesstored in the multi-picture buffer, an identifier indicating a method ofselecting, as a default reference picture, a picture which has displaytime information prior to that of the current picture and whose codingorder is the closest to that of the current picture out of the codedpictures stored in the multi-picture buffer or an identifier indicatinga method of selecting, as a default reference picture, a picture whichhas display time information subsequent to that of the current pictureand whose coding order is the closest to that of the current picture.The coded moving picture data that is generated using this method can bedecoded using the decoding method, having a structure according to thefourth embodiment, which will be explained below.

It is also possible to code the picture number DefRefNo itselfindicating a default reference picture, as in FIG. 7, to code adifferential value between the picture number of the current picture andthe picture number of the picture selected as a default referencepicture, or to code information such as a command for indicating adefault reference picture.

FIG. 13 is a block diagram showing a moving picture coding apparatusused for such case. The default reference picture number generation unit205 outputs the default reference picture number DefRefNo to thevariable length coding unit 206, as shown in FIG. 13. The variablelength coding unit 206 performs variable length coding on the codedresidual data ERes, the prediction type PredType, the reference picturenumber RefNo2, the motion vectors MV1, MV2 as well as the defaultreference picture number DefRefNo and outputs them as coded movingpicture data Str4. The data format for this case is as same as the oneshown in FIG. 7. The coded moving picture data generated using thismethod can be decoded with the use of the decoding method having thestructure described in the second embodiment.

Fourth Embodiment

FIG. 14 is a block diagram showing a moving picture decoding apparatusaccording to the fourth embodiment of the present invention. The samemarks are put for the units and the data operating in the same manner asshown in the block diagram for the moving picture decoding apparatusaccording to the second embodiment in FIG. 8, and the description isabbreviated.

The moving picture decoding apparatus according to the presentembodiment includes a default reference picture number generation unit502 instead of the default reference picture number buffer 402 shown inthe structure of the second embodiment. The variable length decodingunit 501 performs variable length decoding on the inputted coded movingpicture data Str2 and outputs the coded residual data ERes, theprediction type PredType, the reference picture number RefNo2, and themotion vectors MV1, MV2. The default reference picture number generationunit 502 generates a default reference picture number DefRefNo in thesame manner as the default reference picture number generation unit 201described in the third embodiment and outputs, to the motioncompensation unit 403, the default reference picture number DefRefNo asa reference picture number RefNo1.

According to the present embodiment as described above, it is possibleto decode properly the coded moving picture data Str2 which is coded bythe moving picture coding apparatus using the moving picture codingmethod according to the present invention described in the thirdembodiment.

The moving picture decoding apparatus is constructed as below whendecoding the coded moving picture data Str3 in which the IdentifierIdent for indicating a method of selecting a default reference pictureis included, as illustrated in the variation of the third embodimentdescribed above.

FIG. 15 is a block diagram showing the moving picture decoding apparatusused for this case. The variable length decoding unit 503 performsvariable length decoding on the inputted coded moving picture data Str3and outputs the coded residual data ERes, the prediction type PredType,the reference picture number RefNo2, the motion vectors MV1, MV2 as wellas the identifier Ident for indicating a method of selecting a defaultreference picture, as shown in FIG. 15. The default reference picturenumber generation unit 504 generates a default reference picture numberDefRefNo using the method of selecting the default reference picture,indicated by the identifier inputted from the variable length decodingunit 503, and outputs, to the motion compensation unit 403, the defaultreference picture number DefRefNo as a reference picture number RefNo1.

Thus, it is possible to decode properly the coded moving picture dataStr3, in which the identifier Ident for identifying a method ofselecting a default reference picture is included, as described above inthe third embodiment.

Fifth Embodiment

The present embodiment describes coding using a direct mode when codingis performed with reference only to the pictures, each of which hasdisplay order information that is prior to that of the current picture.

FIG. 16 is a diagram illustrating a direct mode using plural referencepictures, each of which has display order information prior to that ofthe current picture, according to the fifth embodiment of the presentinvention. Here, a picture Pic represents a current picture to be coded,pictures Ref1 and Ref2 represent reference pictures, a block Blkrepresents a current block to be coded and a block Blk0 represents ablock in the reference picture Ref1, which is co-locating with thecurrent block Blk. A motion vector MV01 represents a forward referencepicture that is used for coding the block Blk0, a picture Ref3represents a reference picture used by the motion vector MV01, a motionvector MV1 represents a motion vector with respect to the referencepicture Ref1, a motion vector MV2 represents a motion vector withrespect to the reference picture Ref2, a block RefBlk1 represents areference block which is referred to by the motion vector MV1, and ablock RefBlk2 represents a reference block which is referred to by themotion vector MV2.

For the reference pictures, for example, pictures respectively havingdisplay order information which is prior to and is the closest and thesecond closest to that of the current picture are selected from thecoded pictures stored in the multi-picture buffer. In this case,assuming that TRD1 represents a differential value between the displayorder information of the current picture Pic and that of the referencepicture Ref1, TRD2 represents a differential value between the displayorder information of the reference picture Ref1 and that of thereference picture Ref3 and TRD3 represents a differential value betweenthe display order information of the current picture Pic and that of thereference picture Ref2, the motion vectors MV1 and MV2 to be used forcoding the current block can be calculated using the followingequations.

MV1=MV01×(TRD1/TRD2)  (Equation A)

MV2=MV01×(TRD3/TRD2)  (Equation B)

By using the method described above, the reference pictures and themotion vectors in the case of using a direct mode can be determined.

In the H.264 described above, a method for explicitly controllingpictures to be stored in the multi-picture buffer by including controlinformation for storing and removing the coded pictures in and from themulti-picture buffer in the coded moving picture data is discussed.Under such control, there might be a case in which only the pictureshaving display order information subsequent to that of the currentpicture are stored in the multi-picture buffer. The following describesa method to realize a direct mode for a picture to which a pluralreference picture interpolation prediction is applied, when only thepictures having display order information which is subsequent to that ofthe current picture are stored in the multi-picture buffer.

FIG. 17 is an illustration showing a direct mode using plural referencepictures respectively having display order information which issubsequent to that of the current picture, according to the fifthembodiment of the present invention. Here, a picture Pic represents acurrent picture to be coded, pictures Ref1 and Ref2 represent referencepictures, a block Blk represents a current block to be coded, a blockBlk0 represents a block in the reference picture Ref1, co-locating withthe current block Blk. A motion vector MV01 represents a forwardreference motion vector used for coding the block Blk0, a motion vectorMV1 represents a motion vector with respect to the reference pictureRef1 and a motion vector MV2 represents a motion vector with respect tothe reference picture Ref2 whereas a block RefBlk1 represents areference block which is referred to by the motion vector MV1 and ablock RefBlk2 represents a reference block which is referred to by themotion vector MV2.

For the reference pictures, for example, a picture having display orderinformation which is subsequent to and is the closest and the secondclosest to that of the current picture are selected from the codedpictures stored in the multi-picture buffer. In this case, assuming thatTRD1 represents a differential value between the display orderinformation of the current picture Pic and that of the reference pictureRef1, TRD2 represents a differential value between the display orderinformation of the reference picture Ref1 and that of the referencepicture Ref3 and TRD3 represents a differential value between thedisplay order information of the current picture Pic and that of thereference picture Ref2, the motion vectors MV1 and MV2 to be used forcoding the current block can be calculated using the following equations(Equation C) and (Equation D).

MV1=−MV01×(TRD1/TRD2)  (Equation C)

MV2=−MV01×(TRD3/TRD2)  (Equation D)

By using the method described above, the reference pictures and themotion vectors in the case of using a direct mode can be determined.

The processing of the direct mode as described above performed by themoving picture coding apparatus shown in FIG. 6 is executed by themotion estimation unit 101. Similarly, the one performed by the movingpicture decoding apparatus shown in FIG. 8 is executed by the motioncompensation unit 403.

Thus, the moving picture coding apparatus operable for the direct modeas described in the present embodiment allows the use of the direct modeeven when the multi-picture buffer stores only the coded pictures havingdisplay order information that is prior to or subsequent to that of thecurrent picture, and therefore, can improve the coding efficiency sincethe reference pictures and the motion vectors can be omitted. The movingpicture decoding apparatus operable for the direct mode described in thepresent embodiment can decode the coded moving picture data outputted bythe moving picture coding apparatus operable for the direct modedescribed in the present embodiment.

A skip mode can be defined as a case in which coded residual data EResobtained by the inter-picture prediction using the reference picturesand the motion vectors calculated using a direct mode according to thepresent embodiment is “0”. The direct mode according to the presentembodiment allows the use of the direct mode even when the multi-picturebuffer has only the coded pictures having display order informationwhich is prior to or subsequent to that of the current picture,therefore, a skip mode can be selected for such case. The moving picturedecoding apparatus operable for the skip mode described above allows theuse of the skip mode and therefore can improve the coding efficiency.The moving picture decoding apparatus operable for the direct modedescribed in the present embodiment can decode the coded moving picturedata outputted by the moving picture coding apparatus operable for thedirect mode described in the present embodiment.

In the above description for FIGS. 16 and 17, a motion vector withrespect to the reference picture Ref1 can be selected freely and adifferential vector between the motion vector and the motion vector MV2described above can be coded as well. Similarly, a motion vector withrespect to the reference picture Ref2 can be selected freely and adifferential vector between the motion vector and the motion vector MV2described above can be also coded.

In the present embodiment, the skip mode described in the presentembodiment is used when the multi-picture buffer has only the codedpictures having display order information which is prior to orsubsequent to that of the current picture. However, a picture havingdisplay order information which is the closest and the second closest tothat of the current picture may be selected from the pictures stored inthe multi-picture buffer. The procedure may be modified so that the skipmode described in the present embodiment is adapted to the case in whichthe two selected pictures are the pictures having display orderinformation which is prior to or subsequent to that of the currentpicture.

Sixth Embodiment

In the H.264 as described above, a skip mode for a picture to which aplural reference picture interpolation prediction is applied indicatesthat the coded residual data resulted from the inter-picture predictionusing a direct mode is “0”. In contrast, the moving picture codingapparatus and the moving picture decoding apparatus of the presentinvention employ, as a prediction method to be used for a skip mode, aninter-picture prediction using a reference picture having display orderinformation that is the closest to that of the current picture out ofthe coded pictures in the multi-picture buffer.

FIG. 18 is an illustration showing the inter-picture prediction in thecase of using the skip mode according to the sixth embodiment of thepresent invention. Here, a picture Pic represents a current picture tobe coded, a picture Ref1 represents a coded picture having display orderinformation immediately prior to that of the current picture, a pictureRef2 is a coded picture having display order information immediatelysubsequent to that of the current picture, a block Blk is a currentblock to be coded, a motion vector MV1 represents a motion vectorindicating “0” value with respect to the picture Ref1 and a blockRefBlk1 represents a reference block referred to by the motion vectorMV1. Also, TRD1, a differential value between the display orderinformation of the current picture Pic and that of the picture Ref1,shall be smaller than TRD2, a differential value between the displayorder information of the current picture Pic and that of the pictureRef2.

In the present embodiment, a picture having display order informationwhich is the closest to that of the current picture is used as areference picture. In FIG. 18, a picture having display orderinformation which is the closest to that of the current picture is apicture Ref1. The motion vector MV1 with respect to the picture Ref1indicates “0” both in vertical and horizontal components within thepicture and uses the reference block RefBlk1, which is referred to bythe motion vector MV1, as a predictive image. By using such predictionmethod, the reference pictures and the motion vectors are uniquelydetermined by the moving picture coding apparatus and the moving picturedecoding apparatus, therefore, there is no need to include theinformation indicating reference pictures as well as motion vectors inthe coded moving picture data. With the definition of the skip mode asthe case in which the coded residual data obtained as a result of theinter-picture prediction described above is “0”, only the predictiontype indicating a skip mode may be included in the coded data for theblock to which a skip mode is applied and then transmitted.

In the present embodiment, a picture having display order informationwhich is the closest to that of the current picture, out of the codedpictures stored in the multi-picture buffer, is determined as areference picture. However, a picture having display order informationwhich is prior to and the closest to that of the current picture, out ofthe coded pictures in the multi-picture buffer, may be determined as areference picture.

Also, in the present embodiment, a picture having display orderinformation which is the closest to that of the current picture, out ofthe coded pictures stored in the multi-picture buffer, is determined asa reference picture. However, a picture having display order informationwhich is subsequent to and the closest to that of the current picture,out of the coded pictures in the multi-picture buffer, may be alsodetermined as a reference picture.

The display order information of pictures used in each of the aboveembodiments may be either a value indicating the time to display thepictures or information indicating a relative relation in display orderof the pictures.

The picture mentioned above means both a frame and a field: a frame isused for frame coding whereas a field is used for interlace coding(field coding).

In each of the above embodiments, the same processing can be performedeven in the case of interlace coding for coding a picture as two fields,a top field and a bottom field. In the interlace coding, the codingefficiency can be further achieved since the reference picture numberdoubles. In this case, a picture having the same attribute as thecurrent picture may be used as a priority. Namely, when a currentpicture is a top field, a top field is prioritized to be used as apicture indicated by the default reference picture number DefRefNo. Onthe other hand, when a current picture is a bottom field, a bottom fieldis prioritized to be used as a picture indicated by the defaultreference picture number DefRefNo.

Seventh Embodiment

Furthermore, the processing shown in each of the above embodiments canbe carried out easily in an independent computer system by recording aprogram for realizing the picture coding/decoding method described ineach of the above embodiments onto a recording medium such as a flexibledisk or the like.

FIGS. 19A, 19B and 19C are illustrations of a recording medium forrecording a program for realizing the coding/decoding method describedin the above embodiments in the computer system.

FIG. 19B shows a full appearance of a flexible disk, its structure atcross section and the flexible disk itself whereas FIG. 19A shows anexample of a physical format of the flexible disk as a main body of arecording medium. A flexible disk FD is contained in a case F with aplurality of tracks Tr formed concentrically from the periphery to theinside on the surface of the disk, and each track is divided into 16sectors Se in the angular direction. Thus, the program is stored in anarea assigned for it on the flexible disk FD.

FIG. 19C shows a configuration for recording and reproducing the programon the flexible disk FD. When the program is recorded on the flexibledisk FD, the computer system Cs writes in the program via a flexibledisk drive FDD. When the coding apparatus and the decoding apparatus areconstructed in the computer system using the program on the flexibledisk, the program is read out from the flexible disk and thentransferred to the computer system by the flexible disk drive FDD.

The above explanation is made on an assumption that a recording mediumis a flexible disk, but the same processing can also be performed usingan optical disk. In addition, the recording medium is not limited to aflexible disk and an optical disk, but any other medium such as an ICcard and a ROM cassette capable of recording a program can be used.

The following is a description for the applications of the picturecoding/decoding method illustrated in the above-mentioned embodimentsand a system using them.

FIG. 20 is a block diagram showing an overall configuration of a contentsupply system ex100 for realizing content delivery service. The area forproviding communication service is divided into cells of desired size,and cell sites ex107˜ex110, which are fixed wireless stations, areplaced in respective cells.

This content supply system ex100 is connected to apparatuses such as acomputer ex111, a Personal Digital Assistant (PDA) ex112, a cameraex113, a cell phone ex114 and a cell phone with a camera ex115 via, forexample, Internet ex101, an Internet service provider ex102, a telephonenetwork ex104, as well as the cell sites ex107˜ex110.

However, the content supply system ex100 is not limited to theconfiguration shown in FIG. 20 and may be connected to a combination ofany of them. Also, each apparatus may be connected directly to thetelephone network ex104, not through the cell sites ex107˜ex110.

The camera ex113 is an apparatus capable of shooting video such as adigital video camera. The cell phone ex114 may be a cell phone of any ofthe following system: a Personal Digital Communications (PDC) system, aCode Division Multiple Access (CDMA) system, a Wideband-Code DivisionMultiple Access (W-CDMA) system or a Global System for MobileCommunications (GSM) system, a Personal Handyphone System (PHS), or thelike.

A streaming server ex103 is connected to the camera ex113 via thetelephone network ex104 and also the cell site ex109, which realizes alive distribution or the like using the camera ex113 based on the codeddata transmitted from the user. Either of the camera ex113, the serverwhich transmits the data and the like may code the data. The movingpicture data shot by a camera ex116 may be transmitted to the streamingserver ex103 via the computer ex111. In this case, either the cameraex116 or the computer ex111 may code the moving picture data. An LSIex117 included in the computer ex111 and the camera ex116 performs thecoding processing. Software for coding and decoding pictures may beintegrated into any type of recording medium (such as a CD-ROM, aflexible disk and a hard disk) that is a recording medium which isreadable by the computer ex111 or the like. Furthermore, a cell phonewith a camera ex115 may transmit the moving picture data. This movingpicture data is the data coded by the LSI included in the cell phoneex115.

The content supply system ex100 codes contents (such as a music livevideo) shot by a user using the camera ex113, the camera ex116 or thelike in the same way as shown in the above-mentioned embodiments andtransmits them to the streaming server ex103, while the streaming serverex103 makes stream delivery of the content data to the clients at theirrequests. The clients include the computer ex111, the PDA ex112, thecamera ex113, the cell phone ex114 and so on capable of decoding theabove-mentioned coded data. In the content supply system ex100, theclients can thus receive and reproduce the coded data, and can furtherreceive, decode and reproduce the data in real time so as to realizepersonal broadcasting.

When each apparatus in this system performs coding or decoding, thepicture coding apparatus or the picture decoding apparatus shown in theabove-mentioned embodiment can be used.

A cell phone will be explained as an example of such apparatus.

FIG. 21 is a diagram showing the cell phone ex115 using the picturecoding/decoding method explained in the above-mentioned embodiments. Thecell phone ex115 has an antenna ex201 for communicating with the cellsite ex110 via radio waves, a camera unit ex203 such as a CCD cameracapable of shooting moving and still pictures, a display unit ex202 suchas a liquid crystal display for displaying the data such as decodedpictures and the like shot by the camera unit ex203 or received by theantenna ex201, a body unit including a set of operation keys ex204, avoice output unit ex208 such as a speaker for outputting voice, a voiceinput unit ex205 such as a microphone for inputting voice, a recordingmedium ex207 for recording coded or decoded data such as data of movingor still pictures shot by the camera, data of received e-mails and thatof moving or still pictures, and a slot unit ex206 for attaching therecording medium ex207 to the cell phone ex115. The recording mediumex207 stores in itself a flash memory element, a kind of ElectricallyErasable and Programmable Read Only Memory (EEPROM) that is anonvolatile memory electrically erasable from and rewritable to aplastic case such as an SD card.

Next, the cell phone ex115 will be explained with reference to FIG. 22.In the cell phone ex115, a main control unit ex311, designed in order tocontrol overall each unit of the main body which contains the displayunit ex202 as well as the operation keys ex204, is connected mutually toa power supply circuit unit ex310, an operation input control unitex304, a picture coding unit ex312, a camera interface unit ex303, aLiquid Crystal Display (LCD) control unit ex302, a picture decoding unitex309, a multiplexing/demultiplexing unit ex308, a read/write unitex307, a modem circuit unit ex306 and a voice processing unit ex305 viaa synchronous bus ex313.

When a call-end key or a power key is turned ON by a user's operation,the power supply circuit unit ex310 supplies the respective units withpower from a battery pack so as to activate the digital cell phone witha camera ex115 as a ready state.

In the cell phone ex115, the voice processing unit ex305 converts thevoice signals received by the voice input unit ex205 in conversationmode into digital voice data under the control of the main control unitex311 including a CPU, ROM and RAM, the modem circuit unit ex306performs spread spectrum processing for the digital voice data, and thecommunication circuit unit ex301 performs digital-to-analog conversionand frequency conversion for the data, so as to transmit it via theantenna ex201. Also, in the cell phone ex115, the communication circuitunit ex301 amplifies the data received by the antenna ex201 inconversation mode and performs frequency conversion and theanalog-to-digital conversion to the data, the modem circuit unit ex306performs inverse spread spectrum processing of the data, and the voiceprocessing unit ex305 converts it into analog voice data so as to outputit via the voice output unit ex208.

Furthermore, when transmitting an e-mail in data communication mode, thetext data of the e-mail inputted by operating the operation keys ex204of the main body is sent out to the main control unit ex311 via theoperation input control unit ex304. In the main control unit ex311,after the modem circuit unit ex306 performs spread spectrum processingof the text data and the communication circuit unit ex301 performs thedigital-to-analog conversion and the frequency conversion for the textdata, the data is transmitted to the cell site ex110 via the antennaex201.

When picture data is transmitted in data communication mode, the picturedata shot by the camera unit ex203 is supplied to the picture codingunit ex312 via the camera interface unit ex303. When it is nottransmitted, it is also possible to display the picture data shot by thecamera unit ex203 directly on the display unit ex202 via the camerainterface unit ex303 and the LCD control unit ex302.

The picture coding unit ex312, which includes the picture codingapparatus as described for the present invention, compresses and codesthe picture data supplied from the camera unit ex203 using the codingmethod employed by the picture coding apparatus as shown in theembodiments mentioned above so as to transform it into coded image data,and sends it out to the multiplexing/demultiplexing unit ex308. At thistime, the cell phone ex115 sends out the voice received by the voiceinput unit ex205 during the shooting with the camera unit ex203 to themultiplexing/demultiplexing unit ex308 as digital voice data via thevoice processing unit ex305.

The multiplexing/demultiplexing unit ex308 multiplexes the coded imagedata supplied from the picture coding unit ex312 and the voice datasupplied from the voice processing unit ex305, using a predeterminedmethod, then the modem circuit unit ex306 performs spread spectrumprocessing of the multiplexed data obtained as a result of themultiplexing, and lastly the communication circuit unit ex301 performsdigital-to-analog conversion and frequency transform of the data for thetransmission via the antenna ex201.

As for receiving data of a moving picture file which is linked to a Webpage or the like in data communication mode, the modem circuit unitex306 performs inverse spread spectrum processing for the data receivedfrom the cell site ex110 via the antenna ex201, and sends out themultiplexed data obtained as a result of the inverse spread spectrumprocessing.

In order to decode the multiplexed data received via the antenna ex201,the multiplexing/demultiplexing unit ex308 demultiplexes the multiplexeddata into a coded stream of image data and that of voice data, andsupplies the coded image data to the picture decoding unit ex309 and thevoice data to the voice processing unit ex305, respectively via thesynchronous bus ex313.

Next, the picture decoding unit ex309, including the picture decodingapparatus as described in the present invention, decodes the codedstream of the image data using the decoding method corresponding to thecoding method as shown in the above-mentioned embodiments to generatereproduced moving picture data, and supplies this data to the displayunit ex202 via the LCD control unit ex302, and thus the image dataincluded in the moving picture file linked to a Web page, for instance,is displayed. At the same time, the voice processing unit ex305 convertsthe voice data into analog voice data, and supplies this data to thevoice output unit ex208, and thus the voice data included in the movingpicture file linked to a Web page, for instance, is reproduced.

The present invention is not limited to the above-mentioned system sinceground-based or satellite digital broadcasting has been in the newslately and at least either the picture coding apparatus or the picturedecoding apparatus described in the above-mentioned embodiments can beincorporated into a digital broadcasting system as shown in FIG. 23.More specifically, a coded stream of video information is transmittedfrom a broadcast station ex409 to or communicated with a broadcastsatellite ex410 via radio waves. Upon receipt of it, the broadcastsatellite ex410 transmits radio waves for broadcasting. Then, a home-useantenna ex406 with a satellite broadcast reception function receives theradio waves, and a television (receiver) ex401 or a Set Top Box (STB)ex407 decodes a coded bit stream for reproduction. The picture decodingapparatus as shown in the above-mentioned embodiments can be implementedin the reproducing apparatus ex403 for reading out and decoding thecoded stream recorded on a recording medium ex402 such as a CD and aDVD. In this case, the reproduced moving picture signals are displayedon a monitor ex404. It is also conceivable to implement the picturedecoding apparatus in the STB ex407 connected to a cable ex405 for acable television or the antenna ex406 for satellite and/or ground-basedbroadcasting so as to reproduce them on a monitor ex408 of thetelevision ex401. The picture decoding apparatus may be incorporatedinto the television, not in the Set Top Box. Also, a car ex412 having anantenna ex411 can receive signals from the satellite ex410 or the cellsite ex107 for replaying moving picture on a display device such as acar navigation system ex413 set in the car ex412.

Furthermore, the picture coding apparatus as shown in theabove-mentioned embodiments can code picture signals and record them onthe recording medium. As a concrete example, a recorder ex420 such as aDVD recorder for recording picture signals on a DVD disk ex421, a diskrecorder for recording them on a hard disk can be cited. They can berecorded on an SD card ex422. When the recorder ex420 includes thepicture decoding apparatus as shown in the above-mentioned embodiment,the picture signals recorded on the DVD disk ex421 or the SD card ex422can be reproduced for display on the monitor ex408.

As for the structure of the car navigation system ex413, the structurewithout the camera unit ex203, the camera interface unit ex303 and thepicture coding unit ex312, out of the components shown in FIG. 22, isconceivable. The same applies for the computer ex111, the television(receiver) ex401 and others.

In addition, three types of implementations can be conceived for aterminal such as the cell phone ex114: a sending/receiving terminalimplemented with both an encoder and a decoder, a sending terminalimplemented with an encoder only, and a receiving terminal implementedwith a decoder only.

As described above, it is possible to use the picture coding method andthe picture decoding method described in the above-mentioned embodimentsfor any of the above-mentioned apparatuses and systems, and by usingthese methods, the effects described in the above-mentioned embodimentscan be obtained.

From the invention thus described, it will be obvious that theembodiments of the invention may be varied in many ways. Such variationsare not to be regarded as a departure from the spirit and scope of theinvention, and all such modifications as would be obvious to one skilledin the art are intended for inclusion within the scope of the followingclaims.

Thus, as described above in detail, with the moving picture codingmethod according to the present invention, there is no need to select,on a block-by-block basis, a single picture from plural coded picturesfor one reference picture and also there is no need to code thisreference picture on a block-by-block basis, therefore, the efficientcoding can be realized and the processing burden can be reduced.

Using the moving picture decoding method according to the presentinvention also allows the coded moving picture data, which is encodedusing a common reference picture and a reference picture according toeach block and then outputted, to be decoded properly.

INDUSTRIAL APPLICABILITY

Thus, the moving picture coding method and the moving picture decodingmethod according to the present invention is practical as a method ofcoding each picture that composes an input moving picture, with the useof, for example, a cell phone, a DVD apparatus and a personal computer,or the like, outputting it as coded moving picture data and decoding thecoded moving picture data.

1. A picture coding and decoding system which includes a picture codingapparatus which codes a picture on a block basis and a picture decodingapparatus which decodes a coded picture on a block basis, wherein saidpicture coding apparatus includes: a dividing unit operable to divide acurrent picture to be coded into blocks; a first selection unit operableto select, for coding a plural-block image unit made up of a pluralityof blocks, a common reference picture to be commonly referred to, fromamong plural reference pictures, the common reference picture being onlyone reference picture that is selected from among the plural referencepictures and is assigned commonly to each of the plurality of blocks ofthe plural-block image unit; a common information description unitoperable to describe common information which identifies the selectedcommon reference picture, in a common information area for theplural-block image unit such that reference picture identificationinformation for the selected common reference picture can be omitted forat least one of the plurality of blocks of the plural-block image unit,instead of describing, per block, reference picture identificationinformation which identifies the selected common reference picture; afirst predictive image generation unit operable to generate a predictiveimage of a current block to be coded included in the plural-block imageunit, using the selected common reference picture; and a coding unitoperable to code the current block using the predictive image, whereinsaid picture decoding apparatus includes: a common information obtainingunit operable to obtain, for decoding a plural-block image unit made upof a plurality of blocks, common information which identifies a commonreference picture, from only one common information area that isprovided for all of the plurality of blocks, instead of obtaining, perblock, reference picture identification information which identifies areference picture from block data of each of the plurality of blocks,the common reference picture being only one reference picture that isselected from among plural reference pictures and is assigned commonlyto each of the plurality of blocks of the plural-block image unit suchthat reference picture identification information for the commonreference picture can be omitted for at least one of the plurality ofblocks of the plural-block image unit; a second selection unit operableto select, for decoding a plural-block image unit, the common referencepicture to be commonly referred to, from among the plural referencepictures based on the obtained common information; a second predictiveimage generation unit operable to generate a predictive image of acurrent block to be decoded included in the plural-block image unit,using the selected common reference picture; and a decoding unitoperable to decode the current block using the predictive image.